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Abstract 

Factor analysis can be a valuable tool for assessing validity. However, 
researchers using the method must attend to the reliabilities of identified 
factors. The study examined the impacts of item variance on facto,' stability and 
structure. The results suggest that one way factor reliability can be maximized 
is by collecting attitudinal data using an unnumbered graphic scale scored with a 
reasonably large number of scale steps. 
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The value of factor analysis as an aide to measurement has been demonstrated 
in nunerous studies (Thompson & Miller , in press) , although it is also clear that 
factor analytic techniques can be applied inappropriately (Overall, 196M)- Factor 
analytic methods can be appropriately applied in the context of theory regarding 
the structure of measures (Guilford, 1967), as a valuable tool for assessing 
measurement validity (Thompson & Pitts, in press), and in some experimental 
research as a prelude to various analysis of variance procedures and their 
analogues (Morrow & Frankiewicz, 1979). However, as O-onbach (1951, p- 2^7) has 
emphasized, "no factor analysis can be interpreted without some appropriate 
estimate of the magnitude of the error of measurement." Several researchers have 
attempted to formulate ways to maximize factor reliability because the magnitude 
of the contribution of factor analytic results is largely a function of factor 
reliability. 

Two general classes of methods for maximizing these reliabilities have been 
identified. First, efforts can be made to maximize factor reliability after data 
have been collected, i.e., during the analytic process. Alpha factor analysis 
(Kaiser & Caffrey, 1965), for example, represents an effort to extract factors 
which are maximally reliable. Similarly, Hendrickson and Green (1972) have 
explored the effects which Guttman-weighting of item responses has upon 
reliability and factor structure. However, it also also possible to maximize 
factor reliability by attending to reliability considerations during the data 
collection process. These efforts are generally directed toward maximizing the 
stability of the entries in the matrix of associations which is to be factored. 
Although this matrix can take the form of a variance-covariance i.iatrix (Thompson & 
Stapleton, 1980), in practice most researchers analyze instead an inter --item 
correlation matrix. 
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Psychometric Considerations 
The stability of indices of association can be maximized in the same way that 
reliability is most easily maximized, i.e., by attempting to increase the standard 
deviations of the data. As Oonlund (1976, p. 118) explains, since reliability is 
maximized "v*ien individuals [or items] tend to stay in the same relative position 
in a group, from one testing to another, it naturally follows that anything which 
reduces the possibility of shifting positions in the group also contributes to 
larger reliability coefficients [or more stable association estimates]. In this 
case greater differences between the scores of individuals reduce the possibility 
of shifting positions." At least three strategies for maximizing the stability of 
indices of association can be identified: include more items on the measure, 
include more and different kinds of people in the sample, or allow a larger nunber 
of responses for each item. Of course, the three strategies each "allow^' subjects 
to generate data which are more spread out, but do not insure that the subjects 
will actually do so. 

Unfortunately, the first two strategies for maximizing the stability of 
indices of association are not always practical or - desirable. For example, longer 
instrinients can contribute to lower response rates or greater ^ experimental 
mortality. Lengthening an instrument can also lower reliability if fatigue begins 
to affect results. Similarly, including more or different kinds of people in a 
sample may be impractical. However, in some cases the nunber of available 
response alternatives can be increased without placing an overwhelming burden on 
subjects. ^ 

Response alternatives for attitudinal measures can be presented in at least 
two ways. Numerical scales provide subjects with a response key, e.g., "SA" = 
STRONaY AGREE, "A" = AGREE, "D" = DISAGREE, "SD" = STRONGLY DISAGREE. Ihe 
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subjects then circle or write the appropriate response for -each item based upon 
the identified key. However , nunbered graphic scales can also be employed to 
collect attitudinal data. Graphic scales are ^frequently employed with semantic 
differential items, for example, when subjects are asked .to mark through a 
nunbered continuum drawn between two bipolar adjectives. Nunnally (1967, p. 520) 
has suggested that graphic scales should be preferred over nunerical scales, 
because "the presence of a graphic scale probably helps to convey the idea of a 
rating continuum. Second, the graphic scale should lessen clerical errors in 
making ratings." 

However, a third method can be employed to collect attitudinal data in a 
non-ipsative manner. Each subject can be asked to draw a line thrcvOgh an 
unnumbered continuum drawn between two bipolar adjectives. Each subject is 
instructed to draw the line through the continuum at the point which best 
represents the desired response. This last measurement alternative might be 
termed an unnunbered graphic scale and has been employed successfully in previous 
research (Jones, Thompson & Miller, 1980; Thompson, Borgers & Ward, 1978). 

Number of Scale Steps 
Each of the three rating scale techniques requires that a choice be made 
regarding the nunber of scale steps which will be Q:nployed. When nunerical or 
nunbered graphic scales are employed the decision must be made before the data are 
collected, because the scales communicate to the subjects the nunber of steps 
which are being utilized. However, when unnunbered graphic scales are employed 
the researcher has some lattitude in making the choice of the nunber of scale 
steps and conceivably could do so even after the data have been collected. In any 
case, a choice of the number of scale steps must be made at some point for all 
three methods. 
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When maximizing che stability of a matrix of associations is a consideration, 
psychometric theory and previous research (Allison, 1972; Guilford, 195^, 
pp. 289-291) suggest that the advantage lies with using more rather than fewer 
scale steps. More steps allow larger standard deviations to be generated so that 
stability of the association indices is maximized. As Nunnally (1967, p. 521) 
explains, "it is true that, as the nunber of scale points inoreases, the error 
variance increases, but at the same time, the true-score variance Iporeases at an 
even more rapid rate." Thus Guilford (195^, P. 291) suggests that "it may pay in 
some favorable situations to use up to 25 scale divisions." Use of a large nirnber 
of scale steps only becomes undesirable vjhen sv^bjects become confused or irritated 
at being confronted with a cognitivoly o\/3rwhelming nunber of response 
alternatives. 

One advantage of unnanberecl graphic scales is that they can be scored using a 
relatively large number of scale steps. This should result in larger item 
standard deviations, higher item reliabilities, and finally more reliable factors. 
The approach Is not frustrating to subjects because the number of intervals is not 
eitplicitly caorinunicated. 

Of course, the approach presunes that the subjects are all .o'<ing similar and 
fairly refined judgments. There is currently at least incidental evidence that 
this presumption is tenable. Various studies (Brown, 1976, 1977, 1978, 1979; 
Miller, Thompson & Frankiewicz, 1975; Thompson, 1980; Thompson & Miller , 1978) 
have employed an instrument with unntmbered graphic scales which were subsequently 
scored using a 15-unit equal-interval scale. The factor structure? identified in 
these studies have been r«narkably invariant across both culturi:ii groups and 
referents. The structures should have oen less invariant if the solutions 
capitalized on measurement error introduced by idiosyncratic sesle interpretation 
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or an untenable presumption that refined judgments were being made. 

Thus, it is conceivable that larger item standard deviations may be achieved 
by employing unnunbered graphic scales to collect data and subsequently scoring 
the scales using a reasonably large number of scale steps. The purpose of this 
research was to determine empirically whether unnunbered graphic scales scored 
using a larger nunber of scale steps produce more stable factors than do 
unnunbered graphic scales scored using a smaller nunber of scale steps. The study 
also examined vi^at impacts scoring metrics have upon comparability of factor 
structures. 

Method 

Data were obtained by conducting a secondary analysis of results previously 
reported by Thompson (1980). The subjects in the previous study were 235 
inservice teachers. The subjects read summaries of 16 models of teaching. The 
validity of the summaries had been previously examined (Jones, Thompson & Miller, 
1980). Each subject rated preference for each model of teaching on an unnunbered 
graphic scale. 

For the purposes of this methodological inquiry the scales were subsequently 
rescored using five different numbers of scale steps. The scales were scored 
using equal-interval scales with respectively 2, 5, 9, 17, and 33 steps. If 
factor stability and structure are insensitive to item variance, since the five 
data sets were generated by the same subjects on the same instrument, the factors 
produced by the data sets should both be equally reliable and have similar 
structures. 
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The data sets were analyzed using alpha factor analysis (Kaiser & Caffrey, 
1965). The procedure generates maximally reliable factors. The procedure also 
produces estimates of the internal consistency reliability of each factor. 
Although there is some argument regarding the matter (McDonald, 1970, pp. 16-20), 
it has been suggested that the estimates can also be interpreted as 
generalizability coefficients. The average standard deviations for the 16 items 
in the five data sets were respectively .47, 2.94, 3.25, 4.45, and 8.54. The 
nunbers of extracted factors were respectively six, seven, seven, four, and four. 
All factors with eigenvalues greater than one were extracted. 

The alpha coefficients for the factors identified in the analysis are 
reported in Table 1. These coefficients are estimates of the internal consistency 
reliability of each factor. Table 1 also rep)orts the cosines of the angles 
between each factor and the related factor in the solution involving either more 
or fewer scale steps. These cosines are essentially validity coefficients 
(Kaiser, Hunka & Bianchini, 1969); they represent the correlation between two 
factors once solutions have been rotated to a position of best fit. Further 
explanation and a heuristic example of the procedure have been provided by Gorsuch 
(1974, pp. 252-257). 

Insert Table 1 about here. 
Discussion 

The cosines presented in Table 1 suggest that the selection of the nunber of 
scale steps to be employed in a study can affect the structure of the identified 
factors. The solutions involving 17 and 33 scale steps were virtually identical. 
The solutions involving 5 and 9 scale steps were reasonably comparable. Other 
combinations of scale steps produced factor structures which were dramatically 
different. The alpha coefficients presented in Table 1 suggest that the factors' 
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produced frcxn data involving more scale steps tend to be minimally more reliable. 

The fundaonental difference in the solutions is that fewer factors were 
extracted for the data sets involving a larger number of scale steps. 
Furthermore, the factors with the lowest alphas in each solution were more 
reliable for solutions involving more scale steps. There are several reasons to 
believe that the solutions involving more scale steps are more likely to be 
replicable. Ihe solutions produce fewer factors, and parsimony in nunber of 
factors minimizes effects of error and sampling specificity (Peterson, 1955). The 
solutions for data involving more scale steps also produced factors with more 
salient items per factor. As Gorsuch (1974, p. 295) explains, "as the nunber of 
salient variables per factor increases, the rotational position will be more 
uniquely determined and will involve less capitalization on chance. The less 
capitalization on chance [which] occurs, the more replicable the results should 
be." 

It is also important that the solutions for data with 17 and 33 scale steps" 
were so comparable. This result suggests that the researcher need not be 
extraordinarily worried that an overly-specific scoring metric will distort the 
meaning of the identified factors. In short, the results suggest that unnunbered 
graphic scales are a promising method of collecting attitudinal data, particularly 
if the scales are scored with a reasonably large number of scale steps. 
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